Sanity Simulations for Saliency Methods

Saliency methods are a popular class of feature attribution tools that aim to capture a model's predictive reasoning by identifying "important" pixels in an input image. However, the development and adoption of saliency methods are currently hindered by the lack of access to underlying model reasoning, which prevents accurate method evaluation. In this work, we design a synthetic evaluation framework, SMERF, that allows us to perform ground-truth-based evaluation of saliency methods while controlling the underlying complexity of model reasoning. Experimental evaluations via SMERF reveal significant limitations in existing saliency methods, especially given the relative simplicity of SMERF's synthetic evaluation tasks. Moreover, the SMERF benchmarking suite represents a useful tool in the development of new saliency methods to potentially overcome these limitations

Towards a More Rigorous Science of Blindspot Discovery in Image Models

A growing body of work studies Blindspot Discovery Methods ("BDM"s): methods that use an image embedding to find semantically meaningful (i.e., united by a human-understandable concept) subsets of the data where an image classifier performs significantly worse. Motivated by observed gaps in prior work, we introduce a new framework for evaluating BDMs, SpotCheck, that uses synthetic image datasets to train models with known blindspots and a new BDM, PlaneSpot, that uses a 2D image representation. We use SpotCheck to run controlled experiments that identify factors that influence BDM performance (e.g., the number of blindspots in a model, or features used to define the blindspot) and show that PlaneSpot is competitive with and in many cases outperforms existing BDMs. Importantly, we validate these findings by designing additional experiments that use real image data from MS-COCO, a large image benchmark dataset. Our findings suggest several promising directions for future work on BDM design and evaluation. Overall, we hope that the methodology and analyses presented in this work will help facilitate a more rigorous science of blindspot discovery

Seismic reflections from depths of less than two meters

This is the publisher's version, also available electronically from "http://onlinelibrary.wiley.com".Three distinct seismic reflections were obtained from within the upper 2.1 m of flood-plain alluvium in the Arkansas River valley near Great Bend, Kansas. Reflections were observed at depths of 0.63, 1.46, and 2.10 m and confirmed by finite-difference wave-equation modeling. The wavefield was densely sampled by placing geophones at 5-cm intervals, and near-source nonelastic deformation was minimized by using a very small seismic impulse source. For the reflections to be visible within this shallow range, low seismic P-wave velocities (<300 m/s) and high dominant-frequency content of the data (∼450 Hz) were essential. The practical implementation of high-resolution seismic imaging at these depths has the potential to complement ground-penetrating radar (GPR), chiefly in areas where materials exhibiting high electrical conductivity, such as clays, prevent the effective use of GPR. Potential applications of these results exist in hydrogeology and environmental, Quaternary, and neotectonic geology

Planning for population viability on Northern Great Plains national grasslands

Broad-scale information in concert with conservation of individual species must be used to develop conservation priorities and a more integrated ecosystem protection strategy. In 1999 the United States Forest Service initiated an approach for the 1.2 x 106 ha of national grasslands in the Northern Great Plains to fulfill the requirement to maintain viable populations of all native and desirable introduced vertebrate and plant species. The challenge was threefold: 1) develop basic building blocks in the conservation planning approach, 2) apply the approach to national grasslands, and 3) overcome differences that may exist in agency-specific legal and policy requirements. Key assessment components in the approach included a bioregional assessment, coarse-filter analysis, and fine-filter analysis aimed at species considered at-risk. A science team of agency, conservation organization, and university personnel was established to develop the guidelines and standards and other formal procedures for implementation of conservation strategies. Conservation strategies included coarse-filter recommendations to restore the tallgrass, mixed, and shortgrass prairies to conditions that approximate historical ecological processes and landscape patterns, and fine-filter recommendations to address viability needs of individual and multiple species of native animals and plants. Results include a cost-effective approach to conservation planning and recommendations for addressing population viability and biodiversity concerns on national grasslands in the Northern Grea

The impact of stratospheric ozone feedbacks on climate sensitivity estimates

A number of climate modeling studies have shown that differences between typical choices for representing ozone can affect climate change projections. Here we investigate potential climate impacts of a specific ozone representation used in simulations of the Hadley Centre Global Environment Model for the Coupled Model Intercomparison Project Phase 5. The method considers ozone changes only in the troposphere and lower stratosphere and prescribes stratospheric ozone elsewhere. For a standard climate sensitivity simulation, we find that this method leads to significantly increased global warming and specific patterns of regional surface warming compared with a fully interactive atmospheric chemistry setup. We explain this mainly by the suppressed part of the stratospheric ozone changes and the associated alteration of the stratospheric water vapor feedback. This combined effect is modulated by simultaneous cirrus cloud changes. We underline the need to understand better how representations of ozone can affect climate modeling results and, in particular, global and regional climate sensitivity estimates